Ensemble Forecasts Using Rank Histograms

4 Any decision making process that relies on a probabilistic forecast of future events necessarily 5 requires a calibrated forecast. This paper proposes new methods for empirically assessing 6 forecast calibration in a multivariate setting where the probabilistic forecast is given by an 7 ensemble of equally probable forecast scenarios. Multivariate properties are mapped to a single 8 dimension through a pre-rank function and the calibration is subsequently assessed visually 9 through a histogram of the ranks of the observation’s pre-ranks. Average ranking assigns a 10 pre-rank based on the average univariate rank while band depth ranking employs the concept 11 of functional band depth where the centrality of the observation within the forecast ensemble 12 is assessed. Several simulation examples and a case study of temperature forecast trajectories 13 at Berlin Tegel Airport in Germany demonstrate that both multivariate ranking methods can 14 successfully detect various sources of miscalibration and scale efficiently to high dimensional 15 settings. Supplemental material in form of computer code is available online. 16